Relevance Feedback in Content-Based Image Retrieval

Content-Based Image Retrieval (CBIR) systems are required to effectively harness information from ubiquitous image collections. Despite intense research efforts by the multidisciplinary CBIR community since early 1990s, apparently there is a mismatch between these advances and the one that is truly required to bring success to CBIR in the commercial market place. In this paper we provide an overview of approaches to CBIR. Major approaches to improving retrieval effectiveness via relevance feedback in text retrieval systems are discussed. How these relevance feedback techniques have been adopted to CBIR context and their effect on retrieval effectiveness are presented next. The need for test collections in advancing CBIR research is discussed. The paper concludes by pointing out open issues in CBIR and future research direction. 1 Content-Based Image Retrieval (CBIR) Digital images are produced at an ever increasing rate from diverse sources [27, 28, 70]. A contentbased image retrieval (CBIR) system is required to effectively harness information from these image repositories. Content-based retrieval is characterized by the ability of the system to retrieve relevant images based on the visual and semantic contents of images. Interest in CBIR research has begun over a decade ago [32]. Since then there has been explosive interest in the CBIR research. This research has been truly interdisciplinary in nature. As impressive and synergistic this effort may appear to be, however, these investigations didn’t accrue the benefits of cumulative effect — the research didn’t advance by effectively building upon one another’s work. Furthermore, there is a mismatch between these synergistic efforts and the one that is truly required to bring success to CBIR in the commercial market place. There are four major categories of problems related to CBIR [34]: technical, semantic, content, and relativity. Technical problems are related to practical aspects such as image file formats and sizes, compression standards, resolution variables, and bandwidth of image transmission channels. However, recent impressive advances in these areas have significantly alleviated these problems. Semantic or concept-based problems deal with consistency and subjectivity issues in indexing images and subsequent matching of images with user queries. Controlled vocabularies and standards in the form of thesauri and ontologies are used to alleviate these issues. Projects such as the Art and Architecture Thesaurus (http://www.getty.edu/), Iconclass (http://www.iconclass.nl/), The Thesaurus for Graphic Materials I and II (http://www.loc.gov/rr/print/tgm2/, http://www.loc.gov/rr/print/tgm2/), The Consortium for the Computer Interchange of Museum Information (http://www.cimi.org/), The Art Museum Image Consortium (http://www.amico.org/) are attempting to standardize the indexing language and retrieval mechanisms for CBIR. Problems related to content deal with how to model image content to effectively support CBIR. As we will see later, an array of features and abstractions have been used to realize CBIR. However, the existing solutions are rather domain-specific and are not effective for image collections that sport heterogeneous images. Lastly, relativity includes problems concerning the aboutness — thematic